Skip to content

feat(messages): add OnMapTrace for mower firmware (LZMA-wrapped variant)#1567

Open
Beennnn wants to merge 5 commits into
DeebotUniverse:devfrom
Beennnn:feat/parse-mower-getmaptrace-fw-1.15
Open

feat(messages): add OnMapTrace for mower firmware (LZMA-wrapped variant)#1567
Beennnn wants to merge 5 commits into
DeebotUniverse:devfrom
Beennnn:feat/parse-mower-getmaptrace-fw-1.15

Conversation

@Beennnn

@Beennnn Beennnn commented Apr 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Mower firmwares (observed: GOAT A1600 RTK fw 1.15.13) push spontaneous onMapTrace MQTT messages whose body schema is completely different from the existing GetMapTrace response — same NAME, different shape:

{
  "header": {"fwVer": "1.15.13", "tzm": 120, "ts": "...", ...},
  "body": {"data": {
    "mid": "123456789", "batid": "hmfald", "serial": "1",
    "index": "0", "type": "4",
    "info": "<base64 of LZMA1-compressed JSON>",
    "infoSize": 3455
  }}
}

Decompressing the info field yields a small JSON document of the form:

[
  ["5", "0;-11850,-28849;-11800,-28899;...;", "0;-12850,-23699;...;"],
  ["6", "0;-7899,-39700;...;"]
]

— a list of trajectory groups, each holding one or more ;-separated coordinate strings (negative-and-positive ints relative to a map origin). The leading "0" of each segment is an anchor flag, not a coordinate.

The current GetMapTrace._handle_body_data_dict expects traceValue and raises a KeyError, which the legacy fallback in _handle_error_or_analyse catches and logs as WARNING — Could not parse getMapTrace. In one observed setup this produced 217 520 identical warnings in 3 days (~70k/day, ~50/min during mowing).

Change

Add a dedicated OnMapTrace message handler in deebot_client/messages/json/map/__init__.py that:

  1. Detects the new format via the presence of info. If absent, returns ANALYSE (defers to the legacy handler so vacuum firmwares are unaffected).
  2. Decompresses via the existing deebot_client.rs.util.decompress_base64_data helper — which already accommodates the firmware's trimmed 9-byte LZMA header by injecting 4 zero bytes (see src/util.rs::decompress_lzma).
  3. Parses the JSON, drops the leading "0" anchor of each segment, and concatenates the remaining points across groups into the "x,y;x,y;..." shape that downstream consumers (the Map Rust helper in particular) already accept.
  4. Notifies MapTraceEvent using the firmware serial as start so the Map helper does not clear the trace on every push (it only clears when start == 0 — which we never produce here).

Registered alongside the other JSON map messages in _MESSAGES, so it is dispatched before the legacy getMapTrace fallback. Vacuums still go through the legacy path with traceValue unchanged.

Compatibility with companion PRs

This PR alone is sufficient to eliminate the user's specific log storm AND give them a usable trajectory in HA, vs. only suppressing the noise.

Refs

Test plan

  • test_OnMapTrace_decompresses_and_flattens_groups — single-group happy path with mock encoded payload
  • test_OnMapTrace_concatenates_multiple_groups_and_segments — 2 groups × 2-3 segments → flat point string
  • test_OnMapTrace_no_info_field_returns_analyse — defers to legacy handler when info is absent
  • test_OnMapTrace_empty_groups_returns_analyse — no event emitted when there are no points
  • test_OnMapTrace_corrupt_info_returns_analyse[*] (3 cases: invalid b64, too short, non-JSON) — gracefully degrades, no exception escapes
  • test_OnMapTrace_uses_serial_as_event_start — verify MapTraceEvent.start == int(serial)
  • Full suite: 705/705 pass, no regression
  • Real-world: pending HACS release; the captured payloads in our investigation are reproducible (see Disable getMapTrace() for Ecovacs Goat Lawn Mower #1376 comment for sample)

Notes

  • The test file includes a small Ecovacs-format LZMA encoder (_ecovacs_encode) so the round-trip is end-to-end via the real Rust decompressor — no mocking of that path.
  • The implementation drops the leading "0" anchor of each segment. We treat it as a "begin-segment" flag (so concatenated trajectories don't accidentally include (0, ?) as a coordinate). If it should instead be preserved as a path-break marker, happy to adjust in review.
  • For very long traces, the firmware paginates via index (0, 1, …). This PR currently emits one MapTraceEvent per push without aggregating across index values — start = serial is monotonic per cycle, so the Map helper appends them correctly.

Mower firmwares (observed: GOAT A1600 RTK fw 1.15.13) push spontaneous
``onMapTrace`` messages whose body schema is completely different from
the existing ``GetMapTrace`` response:

    {
      "header": {"fwVer": "1.15.13", ...},
      "body": {"data": {
        "mid": "...", "batid": "...", "serial": "1",
        "index": "0", "type": "4",
        "info": "<base64 of LZMA1 compressed JSON>",
        "infoSize": 3455
      }}
    }

The compressed ``info`` field, once decompressed, is a JSON list of
trajectory groups: ``[[group_id, "0;x1,y1;x2,y2;...;", "0;x,y;..."], ...]``
with negative-and-positive integer coordinates (relative to a map origin).

This adds a dedicated ``OnMapTrace`` message handler that:

1. Detects the new format via the presence of ``info``.
2. Decompresses via the existing Rust ``decompress_base64_data`` helper
   (which already handles the firmware's trimmed 9-byte LZMA header).
3. Parses the JSON, drops the leading ``"0"`` anchor of each segment,
   and concatenates the remaining points across groups.
4. Notifies ``MapTraceEvent`` using the firmware ``serial`` as ``start``
   so the ``Map`` Rust helper does not clear the trace on every push.

Registered alongside the other JSON map messages so it is dispatched
*before* the legacy ``getMapTrace`` fallback (which still serves vacuum
firmwares unchanged).

Tests:

- Happy paths (single group, multi-group, multi-segment).
- ``info`` missing → ANALYSE (defer to legacy handler).
- Empty groups → ANALYSE (no event emitted).
- Corrupt ``info`` (invalid base64, too short, decompresses to non-JSON) → ANALYSE (no exception escapes).
- ``serial`` propagates as ``MapTraceEvent.start``.
- Full suite: 705/705 pass, no regression.

Refs:

- DeebotUniverse#1376 (Disable getMapTrace for Goat) — this PR
  is the proper alternative: instead of disabling, the message is now
  parsed and surfaces as a usable trajectory.
- Companion to DeebotUniverse#1565 (skip legacy fallback for mowers) and DeebotUniverse#1566
  (warn-once rate limit). DeebotUniverse#1565 still serves as a safety net for any
  remaining unhandled map messages on mowers.
reniko pushed a commit to reniko/client.py that referenced this pull request May 1, 2026
GetMapTrace V2 format is better handled by upstream PRs DeebotUniverse#1565 and
DeebotUniverse#1567. The xmp9ds test duplicates existing coverage per maintainer
feedback on DeebotUniverse#1564.

https://claude.ai/code/session_01YFYjxwixRZrtjfv1aUfoVQ
Comment thread deebot_client/messages/json/map/__init__.py Fixed
Comment thread deebot_client/messages/json/map/__init__.py Fixed
Comment on lines +32 to +41
def _ecovacs_encode(payload: bytes) -> str:
raw = lzma.compress(
payload,
format=lzma.FORMAT_RAW,
filters=[{"id": lzma.FILTER_LZMA1,
"preset": lzma.PRESET_DEFAULT,
"dict_size": _DICT_SIZE}],
)
header = bytes([0x5D]) + _DICT_SIZE.to_bytes(4, "little") + len(payload).to_bytes(4, "little")
return base64.b64encode(header + raw).decode("ascii")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not encode the data at runtime. Just use the encoded as static string for the test input

Beennnn and others added 2 commits May 6, 2026 00:35
Address review feedback from edenhaus on DeebotUniverse#1567:
- Replace runtime LZMA encoding in tests with pre-computed static
  base64 strings. Test inputs are now constants, not computed at
  test time.
- Remove mid/batid from debug log message to satisfy CodeQL
  "clear-text logging of sensitive information" alert.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mowers don't expose the regular map capability used by vacuums; their
trajectory only comes through MapTraceEvent. Move accumulation,
FIFO cap and SVG rendering into the library so consumers only forward
the event payload and read back an SVG.
@Beennnn

Beennnn commented May 9, 2026

Copy link
Copy Markdown
Contributor Author

Pushed MowerMapTrace (accumulator + SVG renderer) on the same branch.

Addresses @edenhaus' review on home-assistant/core#169590 ("the creation of the map should go into the library"): moves the rendering and FIFO-cap accumulation out of the HA integration and into this lib. 9 unit tests added (tests/test_mower_trace.py). The HA Core PR is now a thin consumer.

Beennnn added a commit to Beennnn/core that referenced this pull request May 9, 2026
Mower devices (Ecovacs GOAT family) do not expose a ``map=`` capability
in their hardware definitions, so the existing ``EcovacsMap`` image
entity is not created for them — yet recent mower firmwares actively
push trajectory points via ``MapTraceEvent``.

This adds a sibling ``EcovacsMowerTraceMap`` image entity that is
created for any device whose ``capabilities.device_type is MOWER``.
It subscribes directly to ``MapTraceEvent`` on the device's event bus,
parses the ``"x,y;x,y;..."`` payload into a list of points, and
renders a simple SVG polyline of the cumulative trajectory.

Implementation notes:

- The trace is accumulated in memory (cap at 5 000 points to bound
  memory; older points are dropped FIFO).
- Y axis is flipped because the mower coordinate frame is bottom-up
  while SVG is top-down.
- ``viewBox`` is computed per push from the bounding box, with a 5%
  padding, so the entire trajectory remains visible regardless of
  garden size.
- ``stroke-width`` scales with the bounding-box width so the line
  remains visible on both small and large lawns.

Requires ``deebot-client>=18.3.0`` (which ships ``OnMapTrace`` —
DeebotUniverse/client.py#1567).

Tests pending — opening as draft for design feedback first.
- json → orjson per TID251 (banned import)
- mower_trace: rename for-loop var to avoid PLW2901 reassignment
- restore `except (TypeError, ValueError):` parens (ruff format had stripped them, breaking Python 3 syntax)
- imports reorganised by ruff
- ruff format applied to test_on_map_trace.py and the map __init__.py

8 unit tests for OnMapTrace still pass.

Addresses CI 'Run prek checks' fail.
@Beennnn Beennnn force-pushed the feat/parse-mower-getmaptrace-fw-1.15 branch from 5d454e6 to 6fed632 Compare May 12, 2026 10:16
@@ -0,0 +1,98 @@
"""Mower trajectory accumulator and SVG renderer.

Mowers (e.g. Ecovacs GOAT family) do not expose the regular ``map``

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use the map capability. MapTraceEvent is also used by the vacuum bots. I also know that mower have a full map, so we should implement that one instead creating a new workaround just for the traces

ruff-format 0.15.11 (pinned in .pre-commit-config.yaml) incorrectly
rewrites `except (TypeError, ValueError):` to `except TypeError, ValueError:`
which is invalid Python 3 syntax. Confirmed reproducible locally with
`uvx ruff@0.15.11 format --diff`. Newer ruff releases are fine.

Wrap the offending block with `# fmt: off` / `# fmt: on` so prek doesn't
strip the tuple parens. The semantics (single int() call, two distinct
exception types caught) are unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants